Natrual voice speech recognition for flight deck applications

ABSTRACT

A computer system receives a voice command and applies one or more contextual filters produce avionics commands based on the voice command. Elements of the voice command are displayed for acceptance by a pilot before being implemented by an avionics system. Individual elements are reevaluated as necessary. Contextual filters include flight paths, flight phase, aircraft location, current weather conditions or information associated with a particular airport.

FIELD OF THE INVENTION

The present invention is directed generally toward avionics systems andmore particularly to speech recognition in avionics systems.

BACKGROUND OF THE INVENTION

Traditional interface devices such as touchscreens, cursor controldevices and knobs all require a pilot to go “heads-down” when changingdata within avionics systems to locate a control, move a cursor to anitem to be changed, select the item, input a new value and accept thechange. Entering data demands gross and fine motor control as well asvisual attention, all of which can distract a pilot.

Voice recognition technology could allow a pilot to input commandchanges, but existing voice recognition technology is not accurate orreliable enough for aircraft avionics applications. In an avionicsapplication, voice recognition technology needs to be able to allow apilot to dictate any International Civil Aviation Organization (ICAO)identifier or common name from navigation databases, but navigationdatabases include many thousands of items making fast, accurate voicerecognition difficult.

Consequently, it would be advantageous if an apparatus existed that issuitable for quickly and accurately translating voice commands in anavionics application.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a novel method andapparatus for quickly and accurately translating voice commands in anavionics application.

In one embodiment of the present invention, a computer system receives avoice command and parses the voice command into a string of words. Thecomputer system identifies a first word, searches a database for thefirst word and identifies a context associated with the first word. Thecomputer system then identifies a second word and searches a databasebased on the context associated with the first word. The computer systemthen constructs a command suitable for an avionics system.

In some embodiments of the present invention, contexts include flightpaths, flight phase, aircraft location, current weather conditions orinformation associated with a particular airport. In additional,contextual filters could include action oriented filters.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the invention claimed. The accompanyingdrawings, which are incorporated in and constitute a part of thespecification, illustrate an embodiment of the invention and togetherwith the general description, serve to explain the principles.

BRIEF DESCRIPTION OF THE DRAWINGS

The numerous advantages of the present invention may be betterunderstood by those skilled in the art by reference to the accompanyingfigures in which:

FIG. 1 shows a block diagram of a computer system useful forimplementing embodiments of the present invention;

FIG. 2 shows a flowchart of one embodiment of the present invention;

FIG. 3 shows a block diagram representation of one exemplary embodimentof the present invention;

FIG. 4 shows a flowchart of one embodiment of the present invention;

FIG. 5 shows a block diagram representation of one environmentalembodiment of the present invention;

FIG. 6 shows an exemplary representation of a user interface useful indynamic filtering;

FIG. 7 shows a flowchart of one embodiment of the present invention;

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the subject matter disclosed,which is illustrated in the accompanying drawings. The scope of theinvention is limited only by the claims; numerous alternatives,modifications and equivalents are encompassed. For the purpose ofclarity, technical material that is known in the technical fieldsrelated to the embodiments has not been described in detail to avoidunnecessarily obscuring the description.

Referring to FIG. 1, a block diagram of a computer system useful forimplementing embodiments of the present invention is shown. In at leastone embodiment, the computer system includes a processor 100, a memory102 connected to the processor 100, a data storage element 104 connectedto the processor 100 and a microphone 108 connected to the processor100. The processor 100 is configured to receive data corresponding to avoice command through the microphone 108, parse the data into one ormore words comprising the voice command and search one or more databasesin the data storage element 104 for the one or more words. The processor100 produces one or more avionics commands based on the voice commands,and relays such avionics commands to one or more avionics systems.

When searching the one or more databases for the one or more words, theprocessor 100 is configured to sort entries in the one or moredatabases, or select a subset of the one or more databases, based on acontext associated with at least one of the one or more words. In atleast one embodiment, a context is determined by dynamic factors such asthe location of an aircraft, a flight plan, one or more weatherconditions or other such factors relevant to an aircraft, airspace orflight parameter.

In another embodiment, the computer system receives the datacorresponding to the voice command through a communications system 106.Likewise, the computer system may receive data pertaining to factorsrelevant to an aircraft, airspace or flight parameter through thecommunications system 106.

The computer system must effectively communicate the avionics commandsas well as a summary of the changes presented. The computer systemtherefore includes a display 110 connected to the processor 100 todisplay this information. Voice syntax may be broken down intoindividual commands. Where the display 110 comprises a touch sensitivedisplay, the processor 100 may display individual avionics commandelements and allow a pilot or co-pilot to select individual commandelements that are incorrect. Key command elements may be highlighted ina particular color indicating the system will take action on thoseitems. Each key command element may have a checkbox next to itindicating that once a pilot accepts the changes, all checked commandswill be executed. The benefit of breaking the string into individualcommands is that complex strings can be accepted in part and reevaluatedwhere necessary, and it creates an opportunity to preview and verifyeach action. The pilot can “activate” voice commands via voice,touchscreen, or pushing a PTR button for a short duration. The displayof the Voice Recognition messages may be a permanent display of the PFDor it could be suppressed until a new voice recognition string isindicated, at which point a voice recognition window may automaticallyspawn from an edge of the display.

Pilot and copilot must be able to see each other's changes to ensurecrew resource management. The display 110 may comprise mirroreddisplays. Both pilots commonly confirm changes with each other due tothe criticality of a correct response.

Referring to FIG. 2, a flowchart of one embodiment of the presentinvention is shown. In at least one embodiment, a computer systemreceives 200 data corresponding to a voice command. The computer systemparses 202 the data into individual words, each word corresponding to aportion of a command string. In one embodiment, the computer systemsearches 206 a voice command database for a recognizable commandcorresponding to a first word. Recognizable commands may include ICAOidentifiers. The recognizable command may be associated with one or morecontextual filters, each of the one or more contextual filters usefulfor performing voice recognition. The computer system may identify 208 acontextual filter based on the recognizable command.

In another embodiment, the computer system receives 212 data pertainingto one or more factors relevant to an aircraft, airspace or flightparameter such as the location of an aircraft, a flight path, one ormore weather conditions or other such factors and uses such data toidentify 204 a dynamic contextual filter for performing voicerecognition. The contextual filter identified 208 by the computer systemmay also be based on the dynamic contextual filter.

Once the computer system finds the a recognizable command through voicerecognition, the computer system sorts 210 the voice command databasebased on the contextual filter or dynamic contextual filter or both andsearches 214 the sorted voice command database for a recognizablecommand corresponding to a second word.

Dynamic contextual filters may include filters based on flight phase(taxiing, airborne, final approach, etc.), proximity to navigationalaids, airports, traffic, weather, terrain or other flight relevantelements, or airport specific properties such as available taxiways at acurrent airport.

Referring to FIG. 3, a block diagram representation of one exemplaryembodiment of the present invention is shown. In one example, a pilotenters a voice command: “Proceed direct-to CID.” The pilot may press abutton to indicate to a computer system that a voice command is beingentered. Upon receiving the voice command, the computer system parsesthe voice command into distinct elements “Proceed,” “direct-to” and“CID.” In at least one embodiment, the computer system identifies afirst word 302, “Proceed,” as an action and searches a database of voicecommands. The database may be an action subset 300 of a complete voicecommands database, or a database sorted to favor action 300 specificcommands.

Once the computer system identifies the first action word 302, thecomputer system identifies a contextual filter to apply to a voicecommand database. Contextual filters may include voice commands closelyassociated with the identified first word 302 or dynamic filters closelyassociated with a factor relevant to an aircraft, airspace or flightparameter separate from the actual voice command or both. For example,where the computer system receives data indicating that an aircraft iscurrently airborne, an action command to “proceed” 302 may indicatedifferent modifiers based on one or more aircraft states such as“airborne,” “on flight path,” “off flight path,” “current heading” orother associated factors. One or more associated factors may be used tosort a voice command database or produce a filtered subset 304 of thevoice command database including grammar highly associated with thefirst word 302. The filtered subset 304 may be sorted according to aweight or rank based on the one or more aircraft states. The computersystem then searches the sorted, filtered voice command database 304 fora second word 306. In the present example, filtering and sortingproduces a filtered, sorted subset 304 where the second word 306 isincluded in the subset 304 and in a sorted position based on a rankingderived from a dynamic contextual filtering process.

Likewise, once the computer system identifies the second word 362, thecomputer system may identify another contextual filter, based on thesecond word 306 or factors relevant to an aircraft, airspace or flightparameter or both, to apply to a voice command database. For example,where the computer system receives data indicating that an aircraft iscurrently airborne, an action modifier such as “direct-to” 306 mayindicate a particular set of potential objects based on one or moreaircraft states such as “on path to destination,” “on path toalternate,” “on path to nearest suitable airport.” One or moreassociated factors may be used to sort a voice command database orproduce a filtered subset 308 of the voice command database includinggrammar highly associated with the second word 306. The filtered subset308 may be sorted according to a weight or rank based on the one or moreaircraft states. The computer system then searches the sorted, filteredvoice command database 308 for a third word 310. In the present example,filtering and sorting produces a filtered, sorted subset 308 where thethird word 310 is included in the subset 308 and in a sorted positionbased on a ranking derived from a dynamic contextual filtering process.In the present example, the computer system may identify an appropriateairport code “CID” 310.

A computer system executing a method according to the present inventionmay thereby perform voice recognition in real time based on dynamicfactors to quickly and accurately determine a command to relay to anavionics system.

Referring to FIG. 4, a flowchart of one embodiment of the presentinvention is shown. In at least one embodiment, a computer systemreceives 400 data corresponding to a voice command. The computer systemparses 402 the data into individual words, each word corresponding to aportion of a command string. In one embodiment, where at least one ofthe words corresponds to an airport identifier, the computer systemidentifies 404 one or more airports based on a flight path of anaircraft. Each of the one or more airports may be dynamically associatedwith a particular function based on a known flight plan and a currentaircraft location. While performing voice recognition, the computersystem identifies 408 a subset of voice command database includingrelevant airports and searches 410 the voice command database for arecognizable command corresponding to an airport.

In at least one embodiment, the computer system may sort or rank 406 theone or more airports based on one or more known properties of the one ormore airports such as proximity or available facilities. Furthermore,the computer system may apply a ranking based on factors such asaircraft condition or weather.

Referring to FIG. 5, a block diagram representation of one environmentalembodiment of the present invention is shown. In one example, anaircraft traveling along a flight path 504 from an origin airport 500 toa destination airport 510 may include an embodiment of the presentinvention. Where an on-board computer system receives an indication ofthe aircraft location 502, the on-board computer system may identify oneor more airports 506, 508, based on the flight path 504. For example,the on-board computer may identify an emergency airport 506 as theclosest serviceable airport to the current aircraft location 502 via anemergency flight path 512. The on-board computer may dynamicallydetermine the emergency airport 506 based on a known aircraft location502 and airport properties, possibly maintained in an on-board computerdatabase. The on-board computer may alter entries in a voice commanddatabase to reflect the status of an airport as an emergency airport 506such that a voice command associated with an emergency situation willplace a high ranking on the emergency airport 506 during voicerecognition.

Likewise, an on-board computer may identify an alternate airport 508 anddynamically determine a direct alternate flight path 514 to thealternate airport 508. The on-board computer may alter entries in avoice command database to reflect the status of an airport as analternate airport 508 and a direct alternate flight path 514 such that avoice command associated with a situation rendering a destinationairport unserviceable will place a high ranking on the alternate airport508 during voice recognition.

Furthermore, an on-board computer may dynamically determine a directdestination flight path 516 to the destination airport 510. The on-boardcomputer may alter entries in a voice command database to reflect thestatus of the direct destination flight path 516 such that a voicecommand associated with a situation rendering the current flight path504 unserviceable will place a high ranking on the direct destinationflight path 516 during voice recognition.

Referring to FIG. 6, an exemplary representation of a user interfaceuseful in dynamic filtering is shown. In at least one embodiment, a userinterface may include a plurality of windows 600, 602, 604, 606. In anavionics computer system utilizing one or more graphical userinterfaces, the computer system may perform dynamic filtering duringvoice recognition based on window 600, 602, 604, 606 of particularinterest. For example, a first window 600 may display representations ofcommon instrumentation such as an altimeter while a second window 602displays a representation and information pertaining to an airport orairspace selected by a pilot. Where each window 600, 602, 604, 606 isassociated with some defined purpose, each window 600, 602, 604, 606 maybe associated with a weighted list potential commands. During voicerecognition operations, the computer system may identify which window600, 602, 604, 606 the pilot is referencing and apply a dynamic filterbased on the weighted list associated with such window. In at least oneembodiment, the computer system identifies which window 600, 602, 604,606 the pilot is referencing by identifying the location of a pointer orother graphical user interface (GUI) selection mechanism. Where thepointer is located within the confines of a particular window 600, 602,604, 606, the computer system may apply the weighted list of commandsassociated with such window 600, 602, 604, 606.

A person skilled in the art may appreciate that weighted lists ofcommands may be associated with GUI structures having finer granularitythan an entire window. For example, where a window includes a mapshowing a plurality of airports, a pointer over any one airport mayimplicate a weighted list of potential commands applicable to featuresof that airport.

Referring to FIG. 7, a flowchart of one embodiment of the presentinvention is shown. An on-board aircraft computer system continuouslymonitors 700 air-traffic control communication frequencies for anaircraft identification code 702 such as a tail number. When the tailnumber is identified 702 as part of an air-traffic controlcommunication, the on-board computer system performs 704 voicerecognition on the command and if possible produces 710 an avionicscommand string. The pilot may then be given the option to accept theavionics command string.

In some cases, the on-board computer may not be able to produce 710 anavionics command based on the air-traffic control communication becauseof the quality of such communication. In that event, the on-boardcomputer system may receive 706 a voice command confirmation from thepilot as the pilot communicates with air-traffic control. The on boardcomputer may then perform 708 voice recognition on the voice commandconfirmation and produce 710 an avionics command based on the voicerecognized confirmation, or some combination of both the voice commandconfirmation and air-traffic control communication.

On on-board computer system may begin voice recognition upon pilotactivation of a “push-to-talk” button. In another embodiment, anaircraft may include a dedicated “push-to-recognize” button. Theon-board computer system may begin voice recognition upon activation ofthe push-to-recognize button. For example, a pilot may push thepush-to-recognize button and say, “Tune Weather”. The on-board computersystem may compare a current location and the search a navigationdatabase for the closest weather station. The on-board computer systemmay display the result to the pilot and execute such command or wait fora pilot confirmation.

It is believed that the present invention and many of its attendantadvantages will be understood by the foregoing description ofembodiments of the present invention, and it will be apparent thatvarious changes may be made in the form, construction, and arrangementof the components thereof without departing from the scope and spirit ofthe invention or without sacrificing all of its material advantages. Theform herein before described being merely an explanatory embodimentthereof, it is the intention of the following claims to encompass andinclude such changes.

What is claimed is:
 1. A computer system comprising: a processor; memoryconnected to the processor; a display connected to the processor; areceiver connected to the processor; a microphone connected to theprocessor; and computer executable program code configured to execute onthe processor, wherein the computer executable program code isconfigured to: continuously monitor air-traffic communicationsfrequencies for an aircraft identification code; receive air trafficcontroller data corresponding to a voice command received from an airtraffic controller through the receiver, associated with the aircraftidentification code; parse the air traffic controller voice data into aplurality of words corresponding to individual command elements;correlate individual voice command elements based on air trafficcontroller data and individual voice command elements based on pilotvoice data; receive pilot voice data corresponding to a pilot voicecommand from the microphone; parse the pilot voice data into a pluralityof words corresponding to individual command elements; produce acontextual filter based on a pointer location in a graphical userinterface, said pointer location associated with a weighted list ofpotential voice commands, a location of the aircraft, a flight plan, andat least one weather condition; produce a dynamic contextual filterbased on a flight phase selected from a list including taxiing,airborne, and final approach; apply the contextual filter and dynamiccontextual filter to one or more databases of command elements to find afirst word of the plurality of words and define a first command elementassociated with the first word; add the first command element to thedynamic contextual filter; associate at least one command elementcorresponding to an airport identification with at least one predefineddescriptor based on a distance from an airport identified by theprocessor, and available airport facilities; produce a plurality ofavionics commands based on the individual command elements; and receiveone or more confirmations, each of the one or more confirmationscorresponding to an avionics command.
 2. The computer system of claim 1,wherein the computer executable program code is further configured to:determine that at least one of the plurality of avionics commands doesnot accurately correspond to an individual command element; and receivea corrected avionics command.
 3. The computer system of claim 1, whereindisplaying the plurality of avionics commands comprises highlighting oneor more commands in a predefined color indicating the computer systemwill take action on the highlighted commands when confirmed.
 4. Thecomputer system of claim 1, wherein the computer executable program codeis further configured to: filter a voice command database based on thecontextual filter; and search for at least one word in the voice commanddatabase to produce the one or more avionics commands.
 5. The computersystem of claim 4, wherein the contextual filter is based on the airportproperty.
 6. An aircraft computer system comprising: a processor; memoryconnected to the processor; an antenna connected to the processor; amicrophone connected to the processor; and computer executable programcode configured to execute on the processor, wherein the computerexecutable program code is configured to: continuously monitorair-traffic communications frequencies for an aircraft identificationcode; receive air traffic controller data corresponding to a voicecommand received from an air traffic controller through the antenna,associated with the aircraft identification code; parse the air trafficcontroller voice data into a plurality of words corresponding toindividual command elements; correlate individual voice command elementsbased on air traffic controller data and individual voice commandelements based on pilot voice data; receive pilot voice datacorresponding to a pilot voice command through the microphone; parse thepilot voice data into a plurality of words corresponding to individualcommand elements; associate at least one command element correspondingto an airport identification with at least one predefined descriptorbased on a distance from an airport identified by the processor, andavailable airport facilities; produce a contextual filter based on apointer location in a graphical user interface, said pointer locationassociated with a weighted list of potential voice commands, a locationof the aircraft, a flight plan, and at least one weather condition;produce a dynamic contextual filter based on a flight phase selectedfrom a list including taxiing, airborne, and final approach; filter avoice command database based on the contextual filter and dynamiccontextual filter; search for at least one word in the voice commanddatabase; and add the at least one word to the dynamic contextualfilter.
 7. The aircraft computer system of claim 6, wherein thecontextual filter is based on the airport property.
 8. The aircraftcomputer system of claim 6, wherein the computer executable program codeis further configured to: produce a plurality of avionics commands basedon the individual command elements; display the plurality of avionicscommands on the display; mirror the plurality of avionics commands on aco-pilot display; and receive one or more confirmations, each of the oneor more confirmations corresponding to an avionics command.
 9. Theaircraft computer system of claim 8, wherein the computer executableprogram code is further configured to: determine that at least one ofthe plurality of avionics commands does not accurately correspond to anindividual command element; and receive a corrected avionics command.10. The aircraft computer system of claim 8, wherein displaying theplurality of avionics commands comprises highlighting one or morecommands in a predefined color indicating the computer system will takeaction on the highlighted commands when confirmed.
 11. A method forproducing avionics commands through voice recognition comprising:continuously monitoring air-traffic communications frequencies for anaircraft identification code; receiving air traffic controller datacorresponding to a voice command received from an air trafficcontroller; parsing the air traffic controller voice data into aplurality of words corresponding to individual command elements;correlating individual voice command elements based on air trafficcontroller data and individual voice command elements based on pilotvoice data; receiving pilot voice data corresponding to a pilot voicecommand; parsing the pilot voice data into a plurality of wordscorresponding to individual command elements; associating at least onecommand element corresponding to an airport identification with at leastone predefined descriptor based on a distance from an airport identifiedby the processor, and available airport facilities; producing acontextual filter based on a pointer location in a graphical userinterface, said pointer location associated with a weighted list ofpotential voice commands, a location of the aircraft, a flight plan, andat least one weather condition; producing a dynamic contextual filterbased on a flight phase selected from a list including taxiing,airborne, and final approach; filtering a voice command database basedon the contextual filter and dynamic contextual filter; searching for atleast one word in the voice command database; and adding the at leastone word to the dynamic contextual filter.
 12. The method of claim 11,wherein the contextual filter is based on at least one of an aircraftlocation, a weather event and an aircraft status.
 13. The method ofclaim 11, further comprising: producing a plurality of avionics commandsbased on the individual command elements; displaying the plurality ofavionics commands; mirroring the plurality of avionics commands on aco-pilot display; and receiving one or more confirmations, each of theone or more confirmations corresponding to an avionics command.
 14. Themethod of claim 13, further comprising: determining that at least one ofthe plurality of avionics commands does not accurately correspond to anindividual command element; and receiving a corrected avionics command.15. The method of claim 13, wherein displaying the plurality of avionicscommands comprises highlighting one or more commands in a predefinedcolor indicating the computer system will take action on the highlightedcommands when confirmed.